Method and apparatus of recovering image

ABSTRACT

The present disclosure relates to a method of recovering an image from which a motion blur is removed and an image recovering apparatus and the image recovering apparatus according to an exemplary embodiment of the present disclosure includes a signal generator which receives at least one of illuminance information and speed information from a sensor to determine a random flickering pattern and a triggering signal based on the received signal, output the determined random flickering pattern to a lighting unit, and output the triggering signal to a camera; and an image processor which receives an image including a motion blur from the camera to recover the received image based on the random flickering pattern determined by the signal generator.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority of Korean Patent Application No.10-2019-0156409 filed on Nov. 29, 2019, in the Korean IntellectualProperty Office, the disclosure of which is incorporated herein byreference.

BACKGROUND Field

The present disclosure relates to a method and an apparatus ofrecovering an image to solve a motion blur from an image obtained bycapturing a moving object.

Description of the Related Art

In the field of image recovering, various methods are being studied tosolve a motion blur caused due to the movement of an object.

In order to solve the motion blur, when a high speed camera with a fastframe rate is utilized, a clear texture image may be acquired. However,an exposure time (or a shutter speed) of a camera shutter is short sothat a brightness of the entire image is dark so that an influence of anoise is relatively increased, which may degrade image quality.

Further, there is a method of utilizing a pattern used to control theshutter when an image is recovered after installing a shutter whichflickers at a high speed in front of a general camera. In this case, ahigh speed operating shutter which is synchronized with a shutter timingof the camera needs to be added so that it is difficult to reduce thesize of the entire system.

Further, when the image is recovered after adding a coded light sourcewhich operates at a high speed separately from the general camera, apattern used to control an illumination device may be utilized. Here, acoded light source may refer to a light source (that is, a lightingdevice) which operates in accordance with a random flickering pattern atthe outside of the camera. In this case, the recovering performance maybe improved at a low illuminance, but in a general illuminanceenvironment, the influence of ambient light other than the coded lightsource is increased so that when the image is recovered with a patternused to control the coded light source, the image may be degraded due tothe added ambient light component.

In the meantime, according to a method of adding a shutter or a lightingdevice, a length of a pattern used to control the shutter or thelighting device may be arbitrarily determined. If a length of thepattern is different from a length of a motion blur, when the image isrecovered in order to match the length of the pattern and the length ofthe motion blur, a process of extending or reducing the image by one ofinterpolation and decimation is inevitably accompanied. Therefore, theimage recovering performance may be degraded.

SUMMARY

An object of the present disclosure is to propose an image recoveringmethod in accordance with surrounding environments by a normal camera, acoded light source, and environment information during a process ofcapturing and acquiring an image. Specifically, according to anexemplary embodiment disclosed in the present disclosure, a motionblurred image acquired by the normal camera is recovered to acquire aclear image.

Technical problems of the present disclosure are not limited to theabove-mentioned technical problem(s), and other technical problem(s),which is (are) not mentioned above, can be clearly understood by thoseskilled in the art from the following descriptions.

The present disclosure relates to a method of recovering an image fromwhich a motion blur is removed and an image recovering apparatus.According to an aspect of the present disclosure, an image recoveringapparatus includes a signal generator which receives at least one ofilluminance information and speed information from a sensor to determinea random flickering pattern and a triggering signal based on thereceived signal, output the determined random flickering pattern to alighting unit, and output the triggering signal to a camera; and animage processor which recovers the received image based on the randomflickering pattern determined by the signal generator when an imageincluding a motion blur is received from the camera.

Desirably, when the signal generator determines the random flickeringpattern and the triggering signal based on the speed information, alength of the random flickering pattern may be determined in accordancewith the speed information.

Desirably, when the signal generator determines the random flickeringpattern and the triggering signal based on the illuminance information,the random flickering pattern which operates the lighting unit may bedetermined based on the influence of the ambient light.

Desirably, the random flickering pattern may be a point spread function(PSF) pattern.

Desirably, the image recovering apparatus may further include a databasewhich stores a plurality of PSF patterns, and the PSF pattern determinedby the signal generator may be one of the plurality of PSF patternsstored in the database.

Desirably, the random flickering pattern output to the lighting unit andthe triggering signal output to the camera may be synchronized.

Desirably, when the illuminance information is received, the imageprocessor may predict a degradation model of the ambient light when thecamera captures an image, based on the received illuminance information,and recover the received image using a prediction result.

Desirably, when the received image includes a plurality of frames, theimage processor may predict a degradation model of the ambient lightwhen the camera captures an image, using the plurality of frames, andrecover the received image based on the prediction result.

Desirably, the image processor may recover the image based on the randomflickering pattern and the pseudo inverse matrix of the received image.

Desirably, the signal generator and the image processor may bephysically spaced apart from each other.

Another aspect of the present disclosure, an image recovering methodincludes: receiving at least one of illuminance information and speedinformation from a sensor by a signal generator of an image recoveringapparatus; determining a random flickering pattern and a triggeringsignal based on the received signal by the signal generator of the imagerecovering apparatus; outputting the determined random flickeringpattern to the lighting unit and outputting the triggering signal to thecamera by the signal generator of the image recovering apparatus; andwhen an image including a motion blur is received from the camera,recovering the received image based on the random flickering pattern bythe image processor of the image recovering apparatus.

Desirably, when the speed information is received from the sensor, thestep of determining a random flickering pattern and a triggering signalmay include a step of determining a length of the random flickeringpattern in accordance with the speed information.

Desirably, when the illuminance information is received from the sensor,the step of determining a random flickering pattern and a triggeringsignal may include a step of determining a random flickering pattern ofthe lighting unit by ambient light.

Desirably, the recovering of the received image may include receivingilluminance information from the sensor, predicting a degradation modelof ambient light when the camera captures an image, based on thereceived illuminance information, and recovering the received imageusing the prediction result.

Desirably, the recovering of the received image may include, when thereceived image includes a plurality of frames, predicting a degradationmodel of ambient light when the camera captures an image, using aplurality of frames, and recovering the received image using theprediction result, by the image processor.

According to an exemplary embodiment of the present disclosure, an imagerecovering apparatus predicts a PSF pattern with a good invertibilitybased on at least one of speed information and illuminance information,generates a random flickering pattern based on the PSF pattern, andcontrols a lighting unit installed at the outside of the camera tooperate with a generated PSF pattern. Further, the image recoveringapparatus acquires and recovers an image which is modulated with thepredicted PSF pattern based on the generated PSF pattern. Therefore, theimage recovering performance may be improved without adding a separatedevice in the camera.

According to an exemplary embodiment of the present disclosure, a camerasuch as an existing CCTV and expensive equipment may be recycled and aperformance of a latest product may be further improved. Further, alighting unit at the outside of the camera is used so that the imagerecovering performance may be improved regardless of the illuminanceenvironment.

The effects of the present disclosure are not limited to theaforementioned effects, and various other effects are included within arange which is obvious to those skilled in the art from the followingdescription.

BRIEF DESCRIPTION OF DRAWINGS

The above and other aspects, features and other advantages of thepresent disclosure will be more clearly understood from the followingdetailed description taken in conjunction with the accompanyingdrawings, in which:

FIG. 1 is a view for explaining an image recovering apparatus of therelated art;

FIGS. 2A and 2B are views for explaining a square wave function and asignal of a flickering pattern which is randomly coded in a spatialdomain and a frequency domain;

FIG. 3 is a view illustrating an image recovering apparatus according toan exemplary embodiment of the present disclosure;

FIG. 4 is a view for explaining a circulant PSF matrix generated by animage recovering apparatus according to an exemplary embodiment of thepresent disclosure;

FIG. 5 is a view for explaining a frequency characteristic of a randomflickering pattern determined by an image recovering apparatus accordingto an exemplary embodiment of the present disclosure;

FIGS. 6A to 6C are views for explaining PSF modeling in accordance witha random flickering pattern by an image recovering apparatus accordingto another exemplary embodiment of the present disclosure;

FIG. 7 is a block diagram illustrating an image recovering systemaccording to an exemplary embodiment of the present disclosure;

FIG. 8 is a block diagram illustrating an image recovering systemaccording to another exemplary embodiment of the present disclosure;

FIG. 9 is a block diagram for explaining an image recovering systemaccording to another exemplary embodiment of the present disclosure;

FIGS. 10A to 10C are views for explaining an image recovered by an imagerecovering apparatus according to an exemplary embodiment of the presentdisclosure; and

FIG. 11 is a flowchart for explaining an image recovering methodaccording to another exemplary embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENT

Those skilled in the art may make various modifications to the presentdisclosure and the present disclosure may have various embodimentsthereof, and thus specific embodiments will be illustrated in thedrawings and described in detail in the detailed description. However,it should be understood that the present disclosure is not limited tothe specific embodiments, but includes all changes, equivalents, oralternatives which are included in the spirit and technical scope of thepresent disclosure. In the description of respective drawings, similarreference numerals designate similar elements.

Terms such as first, second, A, or B may be used to describe variouscomponents but the components are not limited by the above terms. Theabove terms are used only to discriminate one component from the othercomponent. For example, without departing from the scope of the presentdisclosure, a first component may be referred to as a second component,and similarly, a second component may be referred to as a firstcomponent. The term “and/or” includes combinations of a plurality ofrelated elements or any one of the plurality of related elements.

It should be understood that, when it is described that an element is“coupled” or “connected” to another element, the element may be directlycoupled or directly connected to the other element or coupled orconnected to the other element through a third element. In contrast,when it is described that an element is “directly coupled” or “directlyconnected” to another element, it should be understood that no elementis present therebetween.

Terms used in the present application are used only to describe aspecific exemplary embodiment, but are not intended to limit the presentdisclosure. A singular form may include a plural form if there is noclearly opposite meaning in the context. In the present application, itshould be understood that term “include” or “have” indicates that afeature, a number, a step, an operation, a component, a part or thecombination of those described in the specification is present, but doesnot exclude a possibility of presence or addition of one or more otherfeatures, numbers, steps, operations, components, parts or combinations,in advance.

If it is not contrarily defined, all terms used herein includingtechnological or scientific terms have the same meaning as thosegenerally understood by a person with ordinary skill in the art. Termswhich are defined in a generally used dictionary should be interpretedto have the same meaning as the meaning in the context of the relatedart but are not interpreted as an ideally or excessively formal meaningif it is not clearly defined in the present disclosure.

Hereinafter, exemplary embodiments according to the present disclosurewill be described in detail with reference to accompanying drawings.

FIG. 1 is a view for explaining an image recovering apparatus of therelated art.

The image recovering apparatus is a technology of modeling a degradationfunction generated during the image capturing and recovering an originalimage obtained by removing an influence of a motion blur from theblurred image based on the modeling result. The image recoveringapparatus may be one component of the image recovering system 100 andthe image recovering system 100 may further include a camera in additionto the image recovering apparatus.

As illustrated in FIG. 1, if it is assumed to capture a vehicle which isdriven in one direction on the road at a high speed, the motion blur isgenerated in accordance with a time when the shutter is exposed so thatthe image recovering system may acquire a blurred image. In this case,the blurred image g(x, y) may be expressed by the following Equation 1.g(x,y)=H[f(x,y)]+η(x,y)  [Equation 1]

Here, f(x,y) is an original image without having a motion blur, η(x,y)is a noise function, and H is a degradation function. According toEquation 1, when a lot of information about degradation functions H andnoise functions η(x,y) is provided, it is understood that the blurredimage may be recovered to be close to the original image. If it isassumed that the degradation function H is linear and spatiallyinvariant, Equation 1 may be expressed by the following Equation 2.g(x,y)=h(x,y)*f(x,y)+η(x,y)  [Equation 2]

Here, h(x,y) is an expression of a spatial domain of the degradationfunction and * refers to a convolution function. An expression of afrequency domain with regard to this may be expressed by the followingEquation 3.G(u,v)=H(u,v)F(u,v)+N(u,v)  [Equation 3]

H(u,v) is an expression of a frequency domain of the degradationfunction.

The camera stores an image acquired while the shutter is open at everyframe, in an image sensor, and then outputs the image to the imagerecovering apparatus. That is, the blurred image g(x,y) may be expressedby a convolution of the original image f(x,y) and the degradationfunction h(x,y) for a degradation model generated while the shutter ofthe camera is open. Here, the degradation model may vary by a shutter ofthe camera, a movement of the camera, and a movement of the object.

That is, the camera acquires images while the shutter is open so thatthe degradation function h(x,y) may be considered as a square wavefunction by reflecting the shutter of the camera. Accordingly, in therelated art, when the degradation function was modeled, the square wavefunction was utilized.

In the meantime, in Equation 3, an exact value of the noise cannot beknown so that if variables other than a noise factor are summarized withrespect to the original image F(u,v), the variables may be expressedwith Equation 4.

$\begin{matrix}{{F\left( {u,\upsilon} \right)} = \frac{G\left( {u,\upsilon} \right)}{H\left( {u,\upsilon} \right)}} & \left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack\end{matrix}$

In the above Equation, if a variance value of H(u,v) which is a domainexpression of the degradation function is small and a minimum value ofthe signal is large, the frequency characteristic is good, so that therecovering performance may be improved. However, in the case of thedegradation function in which the camera shutter is reflected as asquare wave, a result obtained by converting to the frequency domainincludes 0 and the degradation function is a sinc function type, so thatit is difficult to say that the frequency characteristic is good.Accordingly, a technique with an improved recovering performance ascompared with a traditional method which utilizes a square wave functionis being studied.

FIGS. 2A and 2B are views for explaining a square wave function and asignal of a flickering pattern which is randomly coded in a spatialdomain and a frequency domain.

A typical camera shutter operation may be represented by a square wavefunction consisting of 0 and 1. In the meantime, a model with a goodfrequency characteristic may be a random binary pattern and a modifieduniformly redundant array (MURA) is one of the patterns which are usedin a coded aperture imaging field.

In other words, the degradation function h(x,y) is modulated into apoint spread function with a good frequency characteristic to improvethe recovering performance. If h(x,y) is modulated by a point spreadfunction of the coded light source, the H(u,v) which is a frequencydomain expression of the degradation function may be evaluated by anoptical transfer function (OTF) which is a Fourier transform pair withthe PSF.

In the meantime, FIG. 2A illustrates a square wave function and a randomflickering pattern in a spatial domain and FIG. 2B illustrates twofunctions in a frequency domain.

Referring to FIG. 2B, when an operation of the camera shutter isexpressed by a square wave pattern (box filter) in the spatial domain,it is confirmed that 0 is included in the frequency domain and avariance of the maximum value and the minimum value is large. However,when the operation of the camera shutter is expressed by a randomflickering pattern (coded pattern), as compared with the square wavepattern, the variance of the maximum value and the minimum value in thefrequency domain is small so that it is said that the frequencycharacteristic is good. Accordingly, the image recovering apparatusaccording to an exemplary embodiment of the present disclosure appliesthe random flickering pattern to the camera shutter to be utilized as adegradation model of the acquired image.

In the meantime, as a method for utilizing a random flickering pattern,a method of additionally mounting a separate shutter which operates at ahigh speed at the outside of the camera shutter and a method ofinstalling a light source which operates at a high speed at the outsideof the camera may be considered to be used.

FIG. 3 is a view illustrating an image recovering apparatus according toan exemplary embodiment of the present disclosure.

The image recovering apparatus according to the exemplary embodiment ofthe present disclosure predicts a point spread function (PSF) patternwith a good invertibility for ambient light, applies the randomflickering pattern determined to have a good invertibility of ambientlight as a flickering pattern of a coded light source, and utilizes thesequence of the ambient light which is modulated when the image isrecovered as a degradation model to improve a recovering performance.

The image recovering apparatus according to the exemplary embodimenttransmits a PSF signal to a coded light source and transmits a triggersignal to the camera. The ambient light needs to be flickered whileexposing the shutter of the camera so that two signals are desirablysynchronized.

In the meantime, an image acquired by the camera which operates with asynchronized signal and the coded light source is recovered by the imageprocessing in the image recovering apparatus and specifically, when theimage is recovered, a pseudo inverse matrix may be used, but therecovering method is not limited thereto.

In order to describe an exemplary embodiment which utilizes a pseudoinverse matrix, specifically, when a speed and a direction of a relativemovement between a camera and a scene or a region of interest areconstant, an acquired image i may be defined by a clear image s to berecovered and a PSF pattern A as represented in Equation 5.i=sA+n  [Equation 5]

Here, n represents a noise, which includes all noises (a dark noise, aread noise, etc.) due to a hardware configuration and noises (aquantization error, etc.) due to a software configuration. Even when thespeed and the direction of the motion blur are not constant, themodeling may be performed based on Equation 5. If it is assumed that theinfluence of n is insignificant, the recovered clear image ŝ may berecovered using a pseudo inverse matrix as represented by the followingEquation 6.ŝ=A ⁺ i  [Equation 6]

Here, A is a two-dimensional circulant PSF matrix in which a length anda pattern are optimized by illuminance information and speedinformation. In the image recovering method according to an exemplaryembodiment, the better the invertibility of A^(†) used to recover theimage, the better the recovering performance. The invertibility isdetermined in accordance with a conditional number of the matrix and thelower the conditional number of the matrix A, the better theinvertibility. In other words, the lower the conditional number of thematrix, the better the invertibility of the matrix. When theinvertibility of the matrix used to recover the image is good, the imagerecovering performance is improved. The recovered clear image Ŝ inEquation 6 may be recovered not only by A^(†), but also by anothermethod. For example, Ŝ may be calculated by a conjugate gradient method.

In the meantime, a circulant PSF matrix will be described with referenceto FIG. 4.

FIG. 4 is a view for explaining a circulant PSF matrix generated by animage recovering apparatus according to an exemplary embodiment of thepresent disclosure.

The image recovering apparatus according to the exemplary embodiment mayfurther include a database which stores a plurality of PSF patterns andselect one PSF pattern among the plurality of PSF patterns in accordancewith speed information.

Referring to FIG. 4, the selected PSF circulates for every pixel one byone to become a two-dimensional PSF matrix. A circulating direction(x-axis, y-axis) is defined in accordance with a moving direction of amoving object. For example, when the moving object moves in a y-axisdirection, one-dimensional PSF circulates in an x-axis direction forevery pixel one by one to generate a two-dimensional circulant matrix.

Referring to FIG. 3 again, the image recovering apparatus according tothe exemplary embodiment may generate a PSF pattern based on illuminanceinformation and speed information.

Specifically, the image recovering apparatus according to the exemplaryembodiment utilizes the illuminance information so that the PSF patternmay be generated in consideration of influence of ambient light when theimage is acquired. Accordingly, as compared with the related art, aclearer image may be recovered.

Therefore, the image recovering apparatus according to the exemplaryembodiment may employ a PSF pattern with the best invertibility byadding information of a coded light source to a brightness of theambient light, an operation speed, and a pattern type of the lightsource of the image recovering system.

First, when the coded light source is used, a degradation model when itis influenced by the ambient light is defined as represented by thefollowing Equation 7.i=s(A ₁ +A ₂)+n  [Equation 7]

Here, A₁ is a random binary pattern which controls a coded light source,A₂ is a degradation model of ambient light excluding the influence ofthe coded light source, and A₁ uses a pattern when A₁+A₂ has the bestfrequency domain characteristic. That is, information of A₂ is necessaryto optimize A₁.

In the meantime, information of A₂ may be acquired by various methods.First, there is a method of predicting and optimizing A₂ in the acquiredimage i using a device which is capable of acquiring illuminanceinformation, such as spectrometer.

Alternatively, when it is assumed that an n-th image acquired from thecamera using the coded light source is i_(n), frames before n-th frameare analyzed or the n-th frame is analyzed to optimize ambient lightinformation A₂ excluding the coded light source. The image recoveringapparatus according to the exemplary embodiment may optimize A₂ usingtwo analyzing methods by a device which is capable of acquiringilluminance information and an acquired image i_(r)(r≤n) and thengenerate A₁ thereby.

Further, the image recovering apparatus according to the exemplaryembodiment utilizes speed information so that the PSF pattern may beprepared by predicting a size of the motion blur and the image may berecovered without performing a separate interpolation or decimationprocess. Accordingly, the recovering performance may be improved.

In the acquired image i, when a length of a motion blur of the object isk and a length of the PSF pattern is m, if m≠k and m<k, the image may bedecimated at a rate of m/k or the PSF pattern may be interpolated at arate of k/m. However, when the size of the image or the PSF pattern isadjusted by the interpolation, information of the image or the PSFpattern may be lost or a performance of recovering an original image maybe degraded. Further, according to the image recovering method of therelated art, it is practically difficult to convert a resolution of theimage for every frame according to a size of the motion blur.

However, the image recovering apparatus according to the exemplaryembodiment acquires a speed and distance information of the object inadvance based on the fact that the size of the motion blur and the speedare proportional to each other, so that it is possible to predict howmuch the motion blur is generated in the image.

FIG. 5 is a view for explaining a frequency characteristic of a randomflickering pattern determined by an image recovering apparatus accordingto an exemplary embodiment of the present disclosure.

A graph of FIG. 5 denoted by “box” represents a frequency characteristicwhen the operation of the camera shutter is represented by a squarewave, which shows the same characteristic as the box filter illustratedin FIG. 2B, so that a detailed description thereof will be omitted.

Further, a modulated light pattern of FIG. 5 represents an ambient lightoperation pattern which does not consider the ambient light excludingthe coded light source. However, a real light pattern shows a frequencycharacteristic when the ambient light which is not considered in themodulated light pattern as disclosed in the present disclosure is added.

However, it is difficult to say that the real light pattern of FIG. 5has a better frequency characteristic than the modulated light pattern.This is because when the optimal pattern calculated by theabove-described Equation is applied to an actual situation, thefrequency characteristic may be different from the characteristic whichis assumed when the pattern is calculated and influences on theconditional number which determines the invertibility of the frequency.

Accordingly, in order to generate a real light pattern, the modulatedlight pattern needs to be generated in consideration of the existingambient light. In other words, the performances of the optimal patternsneed to be compared, rather than the performances of the modulated lightpatterns. Therefore, the image recovering apparatus according to theexemplary embodiment of the present disclosure further applies a randomflickering pattern generated in consideration of the influence of theambient light as well as the coded light source to the camera andutilizes the pattern as a degradation model of the acquired image.

FIGS. 6A to 6C are views for explaining PSF modeling in accordance witha random flickering pattern by an image recovering apparatus accordingto another exemplary embodiment of the present disclosure.

In upper graphs of FIGS. 6A to 6C, an x axis denotes a space and a yaxis denotes a PSF pattern. Further, in lower graphs of FIGS. 6A to 6C,an x axis denotes a pixel of the acquired image and a y axis denotes anintensity per pixel.

FIG. 6A illustrates a one-dimensional image acquired when ambient lightexcluding the coded light source is considered, FIG. 6B illustrates aone-dimensional image acquired by a camera when a coded light source isused, but ambient light excluding the coded light source is notconsidered. FIG. 6C illustrates a one-dimensional image when the ambientlight is considered and used.

Referring to FIG. 6C, as compared with FIGS. 6A and 6B, it is confirmedthat the intensity of the image is improved. Accordingly, like the imagerecovering apparatus according to the exemplary embodiment of thepresent disclosure, it is understood that when ambient light is used andthe ambient image is considered, the intensity of the image is improvedso that the recovering performance is also improved.

FIG. 7 is a block diagram illustrating an image recovering systemaccording to an exemplary embodiment of the present disclosure.

First, the image recovering system 700 includes an image recoveringapparatus 710, a sensor 720, a camera 730, and a lighting unit 740.

The sensor 720 senses the environment in the vicinity of the camera 730and includes an illuminance sensor and a speed sensor, but a type ofsensors included in the sensor 720 is not limited thereto.

The camera 730 captures an image for the object and the lighting unit740 receives a random flickering pattern from the image recoveringapparatus 710 to perform the flickering. In the meantime, it is obviousto those skilled in the art that the coded light source described in thespecification is the same component as the lighting unit 740 of FIG. 7.

In the meantime, the image recovering apparatus 710 includes a signalgenerator 711 and an image processor 712 and generates a randomflickering pattern and a triggering signal which control the lightingunit 740 and the camera 730 and receives an image with a motion blur torecover the image as an image from which the influence of the motionblur is removed.

Specifically, the signal generator 711 receives at least one of theilluminance information and the speed information from the sensor 720 todetermine the random flickering pattern and the triggering signal basedon the received signal and output the determined random flickeringpattern to the lighting unit 740 and output the triggering signal to thecamera 730.

Here, when the signal generator 711 determines the random flickeringpattern and the triggering signal based on the speed information, thelength of the random flickering pattern may be determined in accordancewith the speed information. That is, the signal generator 711 predictsthe length of the motion blur which may be generated in the image basedon the speed information in advance to optimize the random flickeringpattern.

Further, when the signal generator 711 determines the random flickeringpattern and the triggering signal based on the illuminance information,the random flickering pattern of the lighting unit may be determinedbased on the influence of the ambient light.

In the meantime, the random flickering pattern may be a point spreadfunction (PSF). In this case, the image recovering apparatus 710 mayfurther include a database which stores a plurality of PSF patterns andthe PSF pattern determined by the signal generator 711 may be one of aplurality of PSF patterns stored in the database.

In the meantime, the random flickering pattern output to the lightingunit 740 and the triggering signal output to the camera 730 may besynchronized.

Further, when the image including a motion blur is received from thecamera, the image processor 712 may recover the received image based onthe random flickering pattern determined by the signal generator.

When the illuminance information is received, the image processor 712predicts a degradation model of the ambient light when the camera 730captures an image, based on the received illuminance information, andrecovers the received image using a prediction result.

When the received image includes a plurality of frames, the imageprocessor 712 predicts a degradation model of the ambient light when thecamera 730 captures an image, using the plurality of frames, andrecovers the received image based on a prediction result.

The image processor 712 may recover the image based on the randomflickering pattern and the pseudo inverse matrix of the received image.

FIG. 8 is a block diagram illustrating an image recovering systemaccording to another exemplary embodiment of the present disclosure.

The signal generator and the image processor of the image recoveringapparatus according to the exemplary embodiment may be physically spacedapart from each other. In the meantime, a sensor, a camera, and alighting unit of FIG. 8 correspond to the sensor 720, the camera 730,and the lighting unit 740 of FIG. 7 so that a detailed descriptionthereof will be omitted.

Referring to FIG. 8, the image processor may be included in a usercomputer 810. In this case, the signal generator and the database whichstores a plurality of random flickering patterns and the image processormay be physically spaced apart from each other. The image processorreceives an image from the camera to recover the image in accordancewith the image recovering method according to the exemplary embodimentsof the present disclosure.

FIG. 9 is a block diagram for explaining an image recovering systemaccording to another exemplary embodiment of the present disclosure.

The image recovering system 900 of FIG. 9 has a structure in which asignal generator (PSF & trigger signal generator) and a database(database of PSF) which stores a PSF pattern are included in a controlboard and an image signal processor is included in a user PC. In themeantime, a sensor (environment), a camera, and a coded light source ofFIG. 9 correspond to the sensor 720, the camera 730, and the lightingunit 740 of FIG. 7 so that a detailed description thereof will beomitted.

The user PC of FIG. 9 further includes an image database, which storesnecessary images when a degradation model of ambient light is predictedbased on the other frame at the time of recovering images, but thefunction of the image database is not limited thereto.

FIGS. 10A to 10C are views for explaining an image recovered by an imagerecovering apparatus according to an exemplary embodiment of the presentdisclosure.

FIGS. 10A to 10C are views illustrating an acquired image and arecovered image by utilizing an image recovering system which includes acoded light source which randomly flickers at a high speed and a camerawith a timing synchronized with the coded light source. This experimentwas performed by capturing an object with a constant moving speed in adark room to control the influence of the illuminance.

FIG. 10A illustrates a 20 fps motion blur image obtained using a speedmeter and it is difficult to identify letters on the object due to themotion blur generated by the movement of the object.

FIG. 10B illustrates an image recovered according to an exemplaryembodiment. Specifically, FIG. 10B illustrates an image which isrecovered by acquiring a modulated image based on the PSF pattern. ThePSF pattern used at this time is“10011001010101101100010001101101000011’ and has a blur size (39 pixels)similar to a predicted length (38 pixels) of the motion blur. Ascompared with FIG. 10A, in the image of FIG. 10B, the letters on theobject may be relatively accurately identified.

In the meantime, FIG. 10C is an image captured by a high speed camerawith a speed of 70 fps. As compared with FIG. 10A, the letters on theobject may be identified better, but it is understood that the image isentirely dark due to the short exposure time of the shutter. In themeantime, as compared with FIG. 10B, the motion blur is significant.

FIG. 11 is a flowchart for explaining an image recovering methodaccording to another exemplary embodiment of the present disclosure.

In step 1110, the signal generator of the image recovering apparatusreceives at least one of illuminance information and speed informationfrom the sensor.

In step 1120, the signal generator of the image recovering apparatus maydetermine a random flickering pattern and a triggering signal based onthe received signal.

Here, when the speed information is received from the sensor, the stepof determining a random flickering pattern and a triggering signal mayinclude a step of determining a length of the random flickering patternin accordance with the speed information.

When the illuminance information is received from the sensor, the stepof determining a random flickering pattern and a triggering signal mayinclude a step of determining a random flickering pattern based on theinfluence of ambient light by the lighting unit.

In step 1130, the signal generator of the image recovering apparatusoutputs the determined random flickering pattern to the lighting unitand outputs the triggering signal to the camera.

In step 1140, when the image processor of the image recovering apparatusreceives an image including a motion blur from the camera, the imageprocessor may recover the received image based on the random flickeringpattern.

Here, the step of recovering the received image may include a step ofreceiving illuminance information from the sensor, a step of predictinga degradation model of ambient light when the camera captures an image,based on the received illuminance information, and a step of recoveringthe received image using the prediction result.

When the received image includes a plurality of frames, the method mayinclude a step of predicting a degradation model of ambient light whenthe camera captures an image using the plurality of frames and a step ofrecovering the received image using the prediction result.

For now, the present disclosure has been described with reference to theexemplary embodiments. It is understood to those skilled in the art thatthe present disclosure may be implemented as a modified form withoutdeparting from an essential characteristic of the present disclosure.Therefore, the disclosed exemplary embodiments may be considered by wayof illustration rather than limitation. The scope of the presentdisclosure is presented not in the above description but in the claimsand it may be interpreted that all differences within an equivalentrange thereto may be included in the present disclosure.

What is claimed is:
 1. An image recovering apparatus, comprising: apattern and triggering signal generator configured to: receive at leastone of illuminance information and speed information of an object from asensor, determine a random flickering pattern and a triggering signalbased on the received information, output the determined randomflickering pattern to a lighting unit, and output the triggering signalto a camera; and an image processor configured to, when an imageincluding a motion blur is received from the camera, recover thereceived image based on the random flickering pattern determined by thepattern and triggering signal generator, wherein the pattern andtriggering signal generator is configured to predict a size of themotion blur based on the received speed information and determine alength of the random flickering pattern in accordance with the predictedsize of the motion blur.
 2. The image recovering apparatus according toclaim 1, wherein when the pattern and triggering signal generatordetermines the random flickering pattern and the triggering signal basedon the speed information, the length of the random flickering pattern isdetermined in accordance with the speed information.
 3. The imagerecovering apparatus according to claim 1, wherein when the pattern andtriggering signal generator determines the random flickering pattern andthe triggering signal based on the illuminance information, the randomflickering pattern which operates the lighting unit is determined basedon influence of ambient light.
 4. The image recovering apparatusaccording to claim 1, wherein the random flickering pattern is a pointspread function (PSF).
 5. The image recovering apparatus according toclaim 4, further comprising: a database which stores a plurality of PSFpatterns, wherein the PSF pattern determined by the pattern andtriggering signal generator is one of the plurality of PSF patternsstored in the database.
 6. The image recovering apparatus according toclaim 1, wherein the random flickering pattern output to the lightingunit and the triggering signal output to the camera are synchronized. 7.The image recovering apparatus according to claim 1, wherein when theilluminance information is received, the image processor predicts adegradation model of ambient light when the camera captures an imagebased on the received illuminance information and recovers the receivedimage using a prediction result.
 8. The image recovering apparatusaccording to claim 1, wherein when the received image includes aplurality of frames, the image processor predicts a degradation model ofambient light when the camera captures an image using the plurality offrames and recovers the received image based on a prediction result. 9.The image recovering apparatus according to claim 1, wherein the imageprocessor recovers an image based on the random flickering pattern and apseudo inverse matrix of the received image.
 10. The image recoveringapparatus according to claim 1, wherein the pattern and triggeringsignal generator and the image processor are physically spaced apartfrom each other.
 11. An image recovering method, comprising: receivingat least one of illuminance information and speed information of anobject from a sensor; determining a random flickering pattern and atriggering signal based on the received information; outputting thedetermined random flickering pattern to a lighting unit and outputtingthe triggering signal to a camera; and recovering, when an imageincluding a motion blur is received from the camera, the received imagebased on the determined random flickering pattern, wherein thedetermining of the random flickering pattern comprises predicting a sizeof the motion blur based on the received speed information anddetermining a length of the random flickering pattern in accordance withthe predicted size of the motion blur.
 12. The image recovering methodaccording to claim 11, wherein the determining of the random flickeringpattern and the triggering signal when speed information is receivedfrom the sensor includes: determining the length of the randomflickering pattern in accordance with the speed information.
 13. Theimage recovering method according to claim 11, wherein the determiningof the random flickering pattern and the triggering signal whenilluminance information is received from the sensor includes:determining the random flickering pattern which operates the lightingunit based on influence of ambient light.
 14. The image recoveringmethod according to claim 11, wherein the recovering of the receivedimage includes: receiving the illuminance information from the sensor;predicting a degradation model of ambient light when the camera capturesan image, based on the received illuminance information; and recoveringthe received image using a prediction result.
 15. The image recoveringmethod according to claim 11, wherein the recovering of the receivedimage includes: predicting a degradation model of ambient light when thecamera captures an image using a plurality of frames when the receivedimage includes the plurality of frames; and recovering the receivedimage using a prediction result.